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1.  INTRODUCTION 


Acquisition  of  scene  properties  such  as  depth  and  surface  orientation  is  one 
of  the  key  problems  in  both  computer  vision  and  robot  vision.  Many  researchers 
have  explored  methods  of  obtaining  these  properties  from  2-D  imagery  using  phy¬ 
sical  constraints  on  the  light  source,  object  class,  or  surface  properties  of  the 
object,  or  using  multiple  views  [lj.  Stereo  vision  is  one  of  the  methods  which 
obtains  depth  information  using  the  disparity  between  the  projected  positions  of 
feature  points  in  two  views  [2].  Finding  the  correspondence  between  feature 
points  in  the  two  stereo  images,  however,  is  the  most  important  and  difficult 
problem  in  stereo  vision  [3,4].  Structured  lighting  methods  are  often  used  for 
quick  acquisition  of  range  information  [5-7].  These  methods  utilize  a  projection 
of  a  regular  pattern  of  light  instead  of  one  of  the  two  stereo  cameras  in  order  to 
encode  3-D  information;  therefore,  establishing  a  correspondence  between  the 
stripes  in  the  image  and  the  light  stripes  in  scene  is  also  necessary  when  a  stripe 
pattern  is  used.  Abrupt  changes  in  depth  are  the  main  concern  in  these  methods 
because  they  disable  the  labeling  of  the  stripe  pattern.  Although  scanning  a  light 
stripe  through  a  slit  has  been  developed  to  avoid  the  correspondence  problem,  it 
is  expensive  and  requires  a  long  time  for  image  acquisition  [8]. 

We  have  proposed  a  new  method  which  determines  the  surface  orientation  of 
each  point  on  an  object,  instead  of  providing  depth  information,  by  projecting  a 
stripe  pattern  without  requiring  correspondence  between  the  stripe  pattern  in  the 
image  and  that  in  the  scene  [9,10].  The  image  acquisition  system  is  arranged 
such  that  all  light  stripes  in  scene  are  almost  parallel  and  the  projection  from 


scene  to  image  is  orthographic.  Thus,  we  can  obtain  a  2(1/2)D  representation  of 
the  scene  by  estimating  the  surface  normals  from  the  slopes  and  intervals  of  the 
stripes  in  the  image.  The  2(1/2)D  image  is  further  segmented  into  planar  or 
singly  curved  surfaces  by  finding  and  examining  clusters  of  the  surface  normals  in 
gradient  space.  Although  the  estimate  of  each  surface  normal  is  somewhat  inac¬ 
curate,  we  can  obtain  much  better  estimates  of  the  geometrical  parameters  of  the 
surfaces  by  utilizing  the  continuity  of  each  surface. 

Sugihara  [11]  proposed  a  method  similar  to  our  approach  which  infers  local 
surface  orientation  using  the  distortion  of  a  known  pattern  projected  on  the 
object  surface  from  a  structured  light  source.  The  projected  texel  on  the  object 
surface  is  detected  and  the  distortion  from  the  regular  pattern  is  measured  in 
order  to  estimate  the  surface  orientation.  Wang  et  al.  [12]  also  proposed  a 
method  which  determines  local  surface  orientation  of  objects  using  grid  coding. 
They  use  two  orthogonal  grid  patterns,  measure  their  orientations  in  the  image, 
and  then  estimate  surface  orientation  at  the  grid  junctions.  Adopting  a  stripe 
pattern  as  the  structured  lighting  provides  us  the  following  advantages  over  these 
methods.  The  computation  time  to  infer  local  surface  orientation  is  shorter  since 
our  method  need  not  find  any  texel  patterns  or  grid  junctions  in  the  image,  but 
need  only  locate  stripe  edge  points  so  that  the  slopes  and  intervals  of  the  stripes 
in  the  image  can  be  easily  measured.  Many  more  surface  normals  are  obtained 
along  the  detected  stripe  edge  points.  The  richness  of  these  points  enables  us  to 
segment  the  image  into  meaningful  regions  corresponding  to  planar  and  singly 
curved  surfaces  and  to  refine  the  surface  parameters  of  these  regions  using  longer 


stripes  and  wider  intervals  between  the  stripes  in  the  image. 


In  this  paper,  we  show  some  applications  of  our  method  and  evaluate  the 
error  in  surface  orientation  introduced  by  the  orthographical  projection.  Evalua¬ 
tion  of  this  error  is  important  in  a  practical  sense.  The  relation  among  the  error 
in  orientation,  the  focal  length  of  the  camera,  the  source  direction  and  the  stripe 
width  is  investigated.  The  following  remarks  can  be  made,  assuming  that  the 
location  of  the  stripe  edge  points  is  precisely  estimated. 

(1)  Obviously,  the  larger  the  focal  length,  the  smaller  the  error  in  orientation. 

(2)  The  stripe  width  does  not  have  much  effect  on  the  estimated  surface  orienta¬ 
tion. 

(3)  The  source  direction  determines  the  observable  and  measurable  field  of  sur¬ 
face  orientation.  Outside  this  area,  estimation  of  the  surface  normals  is  impossi¬ 
ble. 


2.  METHODS 

2.1  Determining  Surface  Orientation 

Surface  orientation  is  determined  locally  from  the  slopes  and  spacings  of  the 
stripes  in  the  image.  Fig.  1  shows  the  geometrical  relations  among  light  planes 
and  an  object  plane  in  the  camera-centered  coordinate  system.  We  define  the 
equations  of  two  light  planes  LP  x  and  LP2  parallel  to  each  other,  and  of  an 
object  plane  OP  on  which  a  stripe  pattern  is  projected,  as 

LPX  :  PaX+Qa  Y  +  Z —Dx 
LP2  :  PtX  +  Qa  Y+Z=D2 


and 


OP  :  PeX+Q0Y+Z  =  C, 

where  {PtfQs)  represents  the  orientation  of  the  light  planes  of  the  stripe  pattern 
projection  and  ( P0,Q0 )  is  the  surface  orientation  to  be  obtained.  The  object 
plane  OP  and  the  light  plane  LP j  (LP 2)  intersect  in  a  line  L  j  (L2).  The  projec¬ 
tions  of  lines  L,  and  L2  onto  the  image  plane  (the  x-y  plane)  are  denoted  by  /, 
and  /2,  respectively.  The  slopes  of  these  lines  (tan0)  and  the  distance  between 
them  (Ax)  in  the  x-axis  direction  are  easily  measured  in  the  image  since  we 
assume  orthographic  projection.  The  geometrical  relation  among  these  variables 
leads  to  the  following  equations: 

P a  P o  D  2~D  j 

tan0= — — — — — ,  Ax  — — — — — . 

7  n  n 


Qa-Qo 


P  -P 
1  6  1  0 


From  these  equations  we  get 


D  2~  D\  Do~D\  .  . 

P0=Pa - T — “  and  Q0=  ?'-+Qs  (2-1) 

Ax  tan(0)*Ax 

That  is,  the  surface  orientation  (P0,Q0)  of  a  point  on  an  object  can  be  locally 
calculated  from  the  slope  (tan0)  and  interval  (Ax)  of  a  stripe  in  the  image. 


2.2  Estimating  the  Locations  of  Stripe  Edge  Points 

The  accuracy  of  estimated  surface  orientation  depends  on  the  measurement 
of  the  slopes  and  spacings  of  the  stripes  in  the  image,  which  are  calculated  from 
stripe  edge  points.  Therefore,  precise  estimation  of  edge  location  is  necessary  in 
our  method.  In  order  to  reduce  the  effect  of  digitizing  error,  the  location  of  an 
edge  point  is  estimated  to  sub-pixel  accuracy  by  using  the  Linear  Mixing  Model 


•V'VW.v.s; 


[13]  of  intensity  around  a  boundary  between  dark  and  bright  regions.  Figs.  2 
show  the  process  of  stripe  edge  location.  Fig.  2(a)  indicates  a  linear  mixing  model 
of  intensity  in  row  I(x),  at  the  center  of  which  a  stripe  edge  exists;  xc  is  the  loca¬ 
tion  of  the  stripe  edge  to  be  obtained.  The  edge  strength  G(x)  in  Fig.  2(c)  is 
obtained  by  applying  a  rectangle  operator  as  in  Fig.  2(b)  to  the  /(x )  in  Fig.  2(a). 
The  following  equation  gives  the  edge  position  xc  as  the  centroid  of  the  peak 
region  in  which  C?(x)  is  greater  than  a  pre-determined  threshold: 

£g(z)»x 

£,G(x) 

i 

In  the  experiments  described  later,  the  input  image  is  digitized  into  256  by  256 
pixels,  256  gray  levels  per  pixel,  and  edge  location  is  represented  to  two  decimal 
places. 

Next,  we  fit  a  line  segment  to  each  of  seven  successive  edge  points.  Those 
segments  whose  fitting  error  exceeds  the  threshold  are  discarded  as  discontinuous 
boundaries.  Thus,  the  slopes  and  intervals  of  the  stripes  in  the  image  are 
obtained  from  the  gradients  and  distances  between  the  fitted  line  segments. 

2.3  Registration  of  Stripe  Pattern 

In  order  to  determine  surface  orientation,  the  first  step  is  to  know  the 
parameters  of  the  stripe  pattern,  that  is,  the  orientation  of  the  light  plane  (Pa, 
Qs)  and  the  width  of  a  stripe  (D2-£>,)  with  respect  to  the  camera-centered  coor¬ 
dinate  system.  We  determine  them  with  a  plaster  sphere  as  follows: 


(1)  Input  an  image  of  the  plaster  sphere  without  a  stripe  pattern  projection  and 
determine  the  location  of  the  sphere  in  the  image  from  its  contour. 

(2)  Input  another  image  of  the  same  sphere  at  the  same  position  with  a  stripe 
pattern  projected  on  it. 

(3)  Apply  the  method  in  Section  2.2  to  the  input  image  in  step  (2)  in  order  to 
measure  the  slopes  and  intervals  of  the  stripes,  and  determine  the  stripe  parame¬ 
ters  by  the  least  squares  method  using  all  the  surface  orientations  calculated  from 
the  image  position  and  the  sphere  location  in  step  (1). 

Fig.  3(a)  shows  the  surface  orientations  used  for  estimation  of  the  stripe  parame¬ 
ters,  and  Fig.  3(b)  shows  the  result  of  inferring  surface  orientations  from  the 
estimated  stripe  parameters.  The  distance  between  the  camera  and  objects  is 
about  4m  (4  degrees  of  visual  angle)  and  the  angle  between  the  optical  axis  and 
the  light  source  direction  is  approximately  15  degrees.  The  standard  deviations 
of  the  estimated  parameters  are  0.3  degrees  in  Pt ,  1.1  degrees  in  Qs  and  1.25  pix¬ 
els  in  ( D2-Dl ),  which  equals  6%  of  the  stripe  width.  With  a  more  perfect  sphere, 
the  standard  deviations  would  be  decreased  since  the  plaster  sphere  used  is  not  a 
perfectly  true  one. 

3.  APPLICATIONS  OF  THE  METHOD 

3.1  Static  Scene 

Finding  a  known  object  and  determining  its  attitude  are  important  and  fre¬ 
quently  occurring  tasks  in  both  computer  vision  and  robot  vision.  In  this  section, 
the  2(l/2)D  image  of  surface  normals  obtained  by  our  method  is  utilized  for  these 


applications. 


Many  objects  observed  in  daily  life  consist  of  planar  or  singly  curved  sur¬ 
faces  the  shapes  of  which  can  be  a  key  feature  for  discrimination  of  such  objects 
as  polyhedra,  cylindrical  objects  and  cone-like  objects.  Therefore,  we  first  map 
the  obtained  surface  normals  onto  gradient  space  in  order  to  segment  the  2(l/2)D 
image  into  planar  or  singly  curved  surfaces.  Since  it  is  difficult  to  extract  the 
exact  shapes  of  planar  regions  from  an  image  with  a  projected  stripe  pattern,  we 
prepare  a  segmented  image  consisting  of  regions  obtained  by  applying  the  Sobel 
operator  to  the  grey-tone  image  of  the  same  scene  with  ordinary  lighting.  The 
surface  normals  to  a  plane  make  a  cluster  in  gradient  space.  By  mapping  them 
from  gradient  space  into  the  2(l/2)D  image  and  the  pre-segmented  image,  we 
obtain  the  planar  region.  Its  orientation  is  precisely  estimated  by  calculating 
again  using  longer  line  segments  and  wider  intervals.  We  can  easily  obtain  fron¬ 
tal  views  of  planar  regions  by  rotating  the  planes  so  that  they  are  parallel  to  the 
image  plane.  Simple  shape  matching  of  planar  regions  to  known  objects  helps  us 
determine  which  object  is  present,  and  its  orientation. 

Figs.  4(a)  and  (b)  show  an  input  image  of  a  cube  with  a  projected  stripe  pat¬ 
tern  and  the  corresponding  edge  picture.  Figs.  4(c)  and  (d)  show  the  needle  map 
of  surface  normals  obtained  by  our  method  and  the  distribution  map  of  the  nor¬ 
mals  in  gradient  space.  There  are  three  clusters  in  Fig.  4(d)  which  correspond  to 
three  regions  A,  B,  and  C  in  the  presegmented  image  Fig.  4(e).  The  frontal 
shapes  of  regions  A,  B  and  C  are  shown  in  Fig.  4(f),  (g)  and  (h),  respectively. 
Fach  of  the  three  angles  between  planes  A,  B  and  C  is  approximately  90  degrees. 


Other  results  can  be  seen  in  [8]. 


3.2  Dynamic  Scene 

Determining  3-D  motion  parameters  in  a  time-varying  scene  is  a  current 
problem  in  computer  vision.  Interesting  theories  have  been  presented  to  estimate 
the  3-D  motions  of  objects  from  a  sequence  of  images  taken  by  a  camera  [14-16]. 
Assuming  the  rigidity  of  objects,  these  theories  analyze  changes  in  the  geometry 
of  an  object’s  images  in  consecutive  frames  to  obtain  3-D  motion  cues.  The 
results  of  applying  these  theories  to  real  scenes,  however,  are  very  sensitive  to 
noise  and  unsatisfactory  in  most  cases.  Therefore,  a  reliable  method  to  obtain 
scene  features  from  each  frame  in  an  image  sequence  is  necessary. 

We  use  the  2(1/2)D  image  of  surface  normals  obtained  by  our  method  as  a 
guide  to  find  the  correspondence  between  frames  in  dynamic  scene  analysis. 
Determining  the  3-D  motion  parameters  of  an  object  is  straightforward,  given  the 
correspondence  of  scene  features  between  consecutive  frames. 

First,  we  map  the  obtained  surface  normals  onto  gradient  space  in  order  to 
segment  the  2(1/2)D  image  into  planar  or  singly  curved  surfaces.  The  surface 
normals  to  a  plane  make  a  cluster  in  gradient  space  as  mentioned  in  Section  3.1. 
By  reverse-mapping  them  from  gradient  space  into  the  2(1/2)D  image,  we  obtain 
the  planar  region.  In  this  case,  the  shape  of  each  region  is  not  extracted  exactly 
because  we  cannot  prepare  the  pre-segmented  image  used  in  the  previous  section 
due  to  the  motions  of  the  objects.  The  surface  normals  to  a  singly  curved  sur¬ 
face,  for  example  a  cylindrical  surface,  make  a  line-like  cluster  in  gradient  space. 


The  line  parameters  obtained  by  fitting  a  line  to  the  cluster  give  us  the  orienta¬ 
tion  of  the  generating  line  of  the  cylindrical  surface.  Similar  refinement  of  the 
surface  parameters  is  performed. 

Figs.  5(a)  and  (b)  show  the  fifth  frame  of  the  sequence  of  input  images  and 
the  moving  objects  in  the  fifth  frame,  respectively.  A  cylinder  and  a  wedge  are 
moving  above  a  stack  of  blocks.  The  process  of  extracting  the  moving  objects  is 
described  in  detail  in  (9).  Fig.  6(a)  shows  the  sampled  surface  normals  on  the 
moving  object  in  the  fifth  frame.  In  Fig.  6(b),  a  histogram  of  these  surface  nor¬ 
mals  in  gradient  space  is  shown.  Since  the  lower  surface  of  the  wedge  is  parallel 
to  the  generating  line  of  the  cylindrical  surface,  the  cluster  corresponding  to  it  is 
merged  into  a  line-like  cluster  (see  Fig.  6(b)).  Therefore,  we  cannot  detect  the 
lower  surface  of  the  wedge  in  gradient  space  at  first.  The  line-like  cluster,  how¬ 
ever,  is  segmented  into  two  surfaces  by  examining  the  surface  continuity  on  the 
2(1/2)D  image.  Fig.  7  shows  the  result  of  the  segmentation. 

Since  each  2(1/2)D  image  is  segmented  into  planar  or  singly  curved  surfaces, 
it  is  easy  to  find  the  correspondences  between  these  surfaces  in  consecutive 
frames.  Finally,  we  determine  the  motion  parameters  from  the  surface  properties 
obtained  for  each  frame  by  assuming  rigidity  of  the  objects  [15].  Fig.  8  shows  the 
changes  of  surface  orientation  in  consecutive  frames  and  the  rotation  axis 
between  two  consecutive  frames  on  the  Gaussian  sphere.  Small  circles  without 
an  axis  indicate  surface  orientations  (surface  normal  for  planar  regions,  and  orien¬ 
tation  of  generating  line  for  cylindrical  surfaces).  Axes  1  and  2  represent  the 
orientation  of  the  rotation  axes  between  the  second  and  fifth  frames  and  t lit*  fifth 


and  eighth  frames,  respectively.  The  angles  between  every  pair  of  regions  in  each 
frame  are  shown  in  Table  1.  The  small  change  in  these  angles  between  frames 
shows  the  validity  of  our  method. 

4.  ERROR  ANALYSIS 

In  the  experiments  in  Section  3,  we  used  about  twenty  times  the  object  size 
as  the  distance  between  the  camera  (or  the  projector)  and  the  object  in  order  to 
reduce  the  approximation  error  involved  in  the  orthographical  projection  that 
was  used.  Such  an  image  acquisition  system  is  more  difficult  to  set  up  when  the 
object  size  is  much  larger  or  the  experimental  room  is  too  small  for  such  a  long 
distance.  Therefore,  investigation  of  the  relations  among  the  surface  orientation 
estimated  by  assuming  orthographical  projection,  the  focal  length,  the  source 
direction  and  the  stripe  width  is  both  interesting  and  meaningful  for  the  use  of 
the  method  in  many  practical  applications. 

Therefore,  let  us  consider  the  effect  on  the  surface  orientation  introduced  by 
approximating  central  projection  by  orthographical  projection.  For  simplicity 
and  without  loss  of  generality,  we  assume  that  Qs ,  the  gradient  of  the  light  plane 
in  the  y-axis  direction,  equals  zero.  Fig.  9  shows  the  top  view  of  the  image 
acquisition  system  in  the  central  projection  system.  In  order  to  compare  with  the 
estimate  based  on  assuming  orthographical  projection,  we  choose  the  virtual 
image  plane  so  that  the  observed  point  O,  on  the  object  lies  on  it.  The  virtual 
slide  plane  through  which  the  light  source  can  project  a  stripe  pattern  in  the 
scene  is  set  at  the  virtual  image  center  and  perpendicular  to  the  source  direction. 


The  equations  and  coordinates  in  the  (2,x)-coordinate  system  are  given  in  Fig.  9. 
The  light  through  the  t-th  slit  5,  on  the  virtual  slide  plane  intersects  the  object 
plane  at  the  point  O,  whose  x-coordinate  is  x,  in  the  virtual  image  plane. 

First,  we  consider  the  effect  on  P0 ,  which  can  be  determined  from  the  inter¬ 
val  between  the  stripes  in  image.  The  error  angle  Ep  between  the  true  orienta¬ 
tion  Pg  and  the  estimated  orientation  is 


E  =  tan  \P0  )- 


tan  ^P/j+tan  *(Pr) 


where  P;  and  Pr  are  obtained  by  substituting  the  left  interval  (x^-x^j)  and  the 
right  interval  (xl+1-x,  )  instead  of  Ax  into  eq.  (2.1),  respectively.  We  show  by 
simulation  how  Ep  would  change  with  various  parameters  (the  orientation  of  the 
object  plane  P0 ,  the  image  position  x, ,  the  focal  length  /,  the  source  direction  Ps 
and  the  stripe  width  (D2-Dy))  since  extracting  the  relation  between  Ep  and  the 
other  parameters  analytically  from  eq.  (4.1)  seems  difficult.  In  the  results  of  the 
following  simulations  <l-4>,  we  assume  that  the  edge  locations  of  the  stripes  in 
the  image  are  obtained  precisely. 

<1  >  P0  vs.  x,(/)  :  First,  we  estimate  the  error  angle  Ep  using  almost  the  same 
parameters  of  focal  length,  source  direction  and  stripe  width  as  in  the  experiment 
in  Section  3.  The  image  size  is  256  by  256  pixels  and  the  equivalent  focal  length 
f  is  3500  pixels,  which  means  about  4  degrees  of  visual  angle.  The  incident  angle 
between  the  optical  axis  and  the  source  direction  is  about  15  degrees  (Ps  =3.68) 
and  the  stripe  width  {D2-Dx)  is  11  pixels.  Fig.  10  shows  the  estimated  value  of 
Ep  at  various  image  positions  (-127<x<128)  and  surface  orientations 


(-63*  <tan“1(P(J  )<64‘).  From  this  figure,  we  can  deduce  the  following  conclu¬ 
sions: 

*  The  error  angle  Ep  is  nearly  proportional  to  the  image  position.  In  other 
words,  we  need  n  times  the  focal  length  in  order  to  reduce  the  error  angle  by  a 

factor  of  — . 

n 

*  The  sign  of  Ep  tells  us  that  the  orientation  is  overestimated  if  the  image  posi¬ 
tion  x  >  0  and  underestimated  if  x  <  0. 

*  Although  the  orientation  at  which  Ep  has  its  maximal  value  depends  on  the 
source  direction  as  described  in  <3>,  the  maximal  value  is  independent  of  it.  In 
this  case,  the  maximal  Ep  (in  short  E pmax)  is  4  degrees  at  the  leftmost  (or  right¬ 
most)  position  in  the  image  when  tan-1(P0  )=-20.0°. 

<2>  (Z?2-^i)  •'  The  change  of  EpmM  with  stripe  width  is  shown  in  Table  1 
where  the  other  parameters  are  the  same  as  in  <1>.  From  this  table,  the  stripe 
width  seems  not  to  have  very  much  effect  on  £’pmax.  However,  there  is  a  trade-off 
problem  between  the  accuracy  and  the  number  of  surface  normals,  because  accu¬ 
rate  detection  of  stripe  edge  position  is  difficult  in  practice.  In  order  to  increase 
the  accuracy  of  the  surface  normal  at  each  point,  we  have  to  project  a  wider 
stripe  pattern,  and  then  we  obtain  a  smaller  number  of  surface  normals.  If  we 
project  a  denser  stripe  pattern,  the  accuracy  of  the  surface  normal  at  each  point 
decreases.  The  optimal  stripe  width  depends  on  the  shape  of  the  object,  the 
albedo  of  the  surface,  and  the  light  source  direction.  We  experimentally  used 
3  6%  of  the  image  size  as  the  width  of  the  stripe  pattern,  and  adopted  a  process 


of  refinement  of  surface  orientation  for  planar  regions  using  longer  line  segments 
and  wider  stripe  intervals. 

<3>  /  vs.  Pa  :  Fig.  11  shows  Ppmax  at  image  position  x,=128  (the  rightmost 
part  of  the  image)  when  we  change  the  focal  length  (/=512n  pixels,  n=l,16) 
and  the  source  direction  (cot_1(P4  )=l+3m,  m  =0,15).  In  this  figure,  the  source 
direction  seems  not  to  have  a  large  effect  on  £’pmaji  although  Epma,x  is  inversely 
proportional  to  the  focal  length  as  described  in  <1>.  The  source  direction  is, 
however,  a  very  important  factor  because  it  determines  the  effective  visual  field. 
If  we  used  a  large  incident  angle  between  the  optical  axis  and  the  source  direc¬ 
tion,  the  range  of  observable  surface  normals  would  be  small. 

<4>  ( P0,Q0 )  :  Finally,  we  estimate  the  total  error  angle  Eepace,  the  difference 
between  the  true  orientation  ( P0,Q0 )  and  the  estimated  orientation  ( Pe,Qe )  in 
space.  Pe  is  an  average  of  Pj  and  Pr ,  and  Qe  is  estimated  from  Pt ,  Pe  and  the 
slope  of  the  stripes  in  the  image,  which  is  calculated  in  the  same  central  projec¬ 
tion  system  as  in  Fig.  9.  Obviously,  Espace  has  its  maximal  value  E™*?e  at  the 
four  corners  of  the  image,  which  are  the  farthest  locations  from  the  image  center. 
Figs.  12  show  the  change  of  E™™(  at  image  position  (128,128)  for  various  object 
orientations:  -63.5°  <tan_1(P0  )< 63.5° ,  and  1"  <tan_1(Q0  )<64” .  (We  need  not 
consider  negative  Q0  because  it  is  symmetric  about  the  origin  since  we  assume 
Qs=  0.)  Figs.  12  (a),  (b),  (c)  and  (d)  are  for  incident  angles  of  5,  10,  15  and  20 
degrees,  respectively,  between  the  optical  axis  and  the  source  direction.  It  is  evi¬ 
dent  from  these  figures  that  the  source  direction  has  a  very  important  role  in 
determining  the  observable  and  measurable  field  of  surface  normals.  Outside  this 


field,  the  calculated  surface  orientations  would  have  serious  errors. 

5.  DISCUSSION 

We  have  described  an  algorithm  which  determines  surface  shape  by  project¬ 
ing  a  stripe  pattern  and  have  evaluated  the  error  in  surface  orientation  intro¬ 
duced  by  assuming  orthographical  projection.  Utilizing  the  results  of  the  error 
estimation,  we  can  compensate  for  the  calculated  surface  orientations  when  the 
focal  length  is  not  very  large.  Since  the  location  of  the  stripe  edge  points  is 
another  important  factor  in  accurate  estimation  of  surface  orientation,  as  men¬ 
tioned  above,  it  is  necessary  to  examine  the  effects  of  surface  orientation,  surface 
albedo  and  light  source  direction  on  edge  location. 

Acquisition  of  range  information  is  possible  in  part  if  we  utilize  the  results  of 
segmentation  of  the  2(l/2)D  image  into  planar  or  singly  curved  surfaces.  Sug- 
ihara  [11]  obtains  not  only  surface  orientation  but  depth  information  by  project¬ 
ing  stripe  patterns  from  two  different  directions.  Stockman  and  Hu  [6]  try  to 
label  the  grid  patterns  using  physical  constraints  on  the  scene  in  order  to  get 
range  information.  In  these  methods,  the  obtained  scene  features  are  too  sparse 
to  segment  the  input  image  into  meaningful  regions  or  to  determine  the  parame¬ 
ters  of  surface  properties.  The  results  of  segmentation  using  our  method  can  be 
used  for  labeling  the  stripe  pattern  inside  each  region,  from  which  we  can  recon¬ 
struct  relative  depth  information  at  the  stripe  edge  points.  Moreover,  we  can 
easily  interpolate  the  depth  information  between  stripe  edge  points  because  the 
surface  of  each  region  has  been  identified  as  planar  or  singly  curved  in  the  seg- 


mentation  process.  Figs.  13(a),  (b)  and  (c)  show  the  results  of  depth  reconstruc¬ 
tion  of  regions  1,  2  and  3  in  Fig.  9,  respectively.  The  depth  computation  is  easy 
since  we  assume  orthographical  projection.  The  standard  deviations  of  range  are 
0.5  pixels  for  the  planar  surfaces  (region  1  and  3)  and  0.7  pixels  for  the  cylindrical 
surface  (region  2).  The  variation  of  reconstructed  range  on  the  surfaces  in  Figs. 
12  is  due  to  the  difference  of  surface  albedo  between  the  plaster  used  for  registra¬ 
tion  of  the  stripe  pattern  and  the  objects,  which  are  made  from  wood  and 
painted  red  (cylinder)  or  blue  (wedge).  However,  we  cannot  reconstruct  the  depth 
relation  between  two  regions  unless  there  is  no  abrupt  change  in  depth  which 
causes  a  break  in  the  stripe  pattern  between  these  regions. 

Shadow  information  can  be  a  cue  to  extracting  the  depth  relation  between 
two  regions  which  are  discontinuous  in  depth.  Using  the  projections  of  the  light 
lines  onto  the  image  plane,  which  are  equivalent  to  the  epipolar  lines  in  stereo 
vision,  we  can  label  each  stripe  in  the  image  of  two  surfaces  as  shown  in  Fig.  14. 
This  displays  the  upper  surface  of  the  wedge  and  its  shadow  on  the  back  wall  in 
Fig.  5  (a).  The  relative  distance  between  them  can  be  obtained  from  the  equa¬ 
tions  of  the  light  stripes  in  the  scene.  The  physical  constraints  on  the  shadow, 
the  number  of  stripes  in  the  scene  and  the  surface  continuity  might  not  be 
sufficient  for  the  unique  labeling  of  the  light  stripes,  but  can  reduce  the  number 
of  candidates,  so  that  some  3-D  structure  of  the  scene  can  be  calculated  from  the 
results  of  consistent  labeling. 
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Fig.  7  Results  of  segmentation 
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Fig.  8  Changes  of  surface  orientation  and  rotation  axes 


Table  1.  Angles  between  the  obtained  surfaces 
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Fig.  9  Top  view  of  image  acquisition  system  in  the  central  projection 


Table  2.  Error  angle  in  orientation  II  (Dr,-D ,) 
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Fig.  10  Error  angle  in  orientation  I  {P0  vs.  a :,(/)) 
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Fig.  11  Error  angle  in  orientation  111  (/  vs.  Ps ) 


